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Abstract. We study the three state toric homogeneous Markov chain 
model and three special cases of it, namely: (i) when the initial state 
parameters are constant, (ii) without self-loops, and (iii) when both 
cases are satisfied at the same time. Using as a key tool a directed 
multigraph associated to the model, the state-graph, we give a bound on 
the number of vertices of the polytope associated to the model which 
does not depend on the time. Based on our computations, we also 
conjecture the stabilization of the f-vector of the polytope, analyze the 
normality of the semigroup, give conjectural bounds on the degree of 
the Markov bases. 



1. Introduction 

In this paper, we consider a discrete time Markov chain Xt, with t = 
1, . . . , T (T > 3), over a finite space of states [S] = {1, . . . , S}. Let w = 
(si, . . . , St) be a path of length T on states [S], which is sometimes written 
as w = (si • • • St) or simply w = si • • • st- We are interested in Markov 
bases of toric ideals arising from the following statistical models 



(1-1) P(W) = C7si/3si,S2 • • • PsT-UST- 

where c is a normalizing constant, indicates the probability of the 
initial state, and (3si,s- are the transition probabilities from state Si to sj. 



The model ( 1.1 ) is called a toric homogeneous Markov chain (THMC) model. 

Commonly in practice, it is important to consider the case where the 
initial parameters are constant; this is, when 71 =72 = • • • = 75; we 
refer to this case as the THMC model without initial parameters. Another 
simplification that arise from practice is when we consider only the transition 
probability between two different states, i.e. when /3si,sj = whenever 
Si = sj; this situation is called a THMC model without self-loops. 

In order to simply the notation throughout this paper we refer to them as 
Model Q, Model Q, Model (|d]), and Model (|c]), according to the following: 



(a): THMC model (1.1) 



(b): THMC model without initial parameters: when 71 = • • • = 75 

Key words and phrases. Markov bases, time homogeneous Markov chains, polyhedrons, 
semigroups. 
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(c) : THMC model without self-loops: f3si,sj = whenever Si = sj. 

(d) : THMC model without initial parameters and without self-loops, i.e., 

both (b) and (c) are satisfied 

In 2010, Hara and Takemura[5] gave a complete description of a Markov 
basis for Model Q, when [S] = {1,2} and T is arbitray, and also for the 
case when T = 3 and [S] is arbitrary. In their next paper |6J, the authors 
provided a Markov basis for Model Q, when [S] = {1,2}, T arbitrary. 
In these articles, all moves found were of degree four or less, regardless 
of the value T. Motivated by these results, we studied Markov bases of 
Models Q - Q. Specifically, we are interested in showing that the degree 
of a minimum Markov basis is bounded when S is fixed and T is arbitrary. 
Each model has an associated design matrix (defined in Section [2]) which 
translates observed data w = (si, . . . , st) into the sufficient statistic. The 
sufficient statistic are the number of transitions from states i to j, for all 
i,j G [S], and for Model ([a]) and ([c|, it also includes the initial state. This 
paper is organized as follows; in Section [2] we describe the design matrices 
for the above models and introduce the state graph, a useful tool we use 
throughout the paper. In Section [4] that the semigroups generated by the 
columns of the design matrices for Model ([a]) and Model ([b| are not normal. 
We also provide computational evidence that for Model and Model ([d]) 
the corresponding semigroups are normal, and conjecture that this holds in 
general. 

In Section |3] we study the properties of the Smith normal form of the 
design matrix and we use some of these results in Section [5j to show the 
following for Model (|d]). 

Theorem 1.1. Let S = 3. The number of vertices of is bounded by 
some constant C which does not depend on T. 

Given the above theorem and our normality conjecture for Model Q, one 
can prove the following conjecture: 

Conjecture 1.2. We consider Model ([d]). Then for 5 = 3 and for any 

T > 4, a minimum Markov basis for the toric ideal ly^^ consists of binomials 
of degree less than or equal to d = Q. Moreover, there are only finitely many 
moves up to a certain shift equivalence relation. 

Additionally for Model ([d]) , we present in Section [g] some of our exper- 
imental results that suggest that the /-vector of the polytope defined by 
the design matrix stabilized (periodically) indepently of T. Our results also 
suggest that for S > 3, the bound d of Conjecture |1.2| depend linearly on S; 



we present these conjectures formally in Section [7 

2. Notation 

Let [S]'^ be the set of all words of length T on states [S]. Similarly let 
[S]^^ be the set of all words of length T on states [S] such that every word 
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has no self-loops; that is, if w = {si, . . . , st) S ["SJ^^^ then Si / Sj+i for 
i = 1, . . . , T — 1. We define V*{[S]'^) to be the set of all multisets of words 
in [S]'^. Similarly, we define 'P*{[S]'Jfi) to be the set of all multisets of words 

in [S]%L- 

Let V([5]"^) be the real vector space with free basis [S]'^ and similarly 
let V ([5]^^) be the real vector space with free basis [5"]^^. Note that 
V {[Sf) ^ R^^ and V {[S]%^) ^ M'5(S'-i)^. We recall some definitions from 
the classic paper of Pachter and Sturmfels[8j. Let A = {aij) be a non- 
negative integer d x m matrix with the property that all column sums are 
equal: 

d d 

i=l i=l 

Let A = [ai a2 ■ ■ ■ am] where aj are the column vectors of A and define 
0"-j = Y[i=i ^i'^ for j = 1, . . . ,m. The toric model of A is the image of the 
orthant M>g under the map 

Here we have d parameters = {0i, . . . , 9d) and a discrete state space of size 
m. In general, the discrete space will be the set of all possible words on [5] of 
length T and we can think of , . . . , 0^ as the logarithm of the probabilities 
7si ) Psi S2" ' PsT-i ST- Below we specify this relation for Models Q, Q, (|c]). 



d 

^ ^ O-irn- 
i=l 



2.1. Model Q. Consider the state space [5*]-^. Model Q is parametrized 
by 7i, . . . ,75 and /Jn, /3i2, . . . , Pss] thus, it is parametrized by all positive 
real vectors of length S and all positive S x S real matrices. Thus, the 
number of parameters is d = S + S'^ and the number of transition of states 
is m = S*^. This toric model is represented by the {S + S"^) x 5""^ matrix 
A^rp, whose rows are indexed by [S] U [S]"^ and the columns are indexed by 
words [S]'^, and it is defined as follows. 

(1) The entry of indexed by row s G [S] and column w = {si, . . . , st) & 
[Sf^ is 1 if s = si and else. 

(2) In row fTicJ2 G [S]"^ and column w = {si, . . . , st) G [S]'^, the entry of 
A^rp is equal to | { i G {1, . . . , T — 1} | aia2 = SiSj+i } |- 



Example 2.1. Ordering [S] U [S]"^ and [S]'^ lexicographically, the matrix 
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2.2. Model ([b]). Similarly, Model Q is parametrized by all positive S x S 
real matrices, as it is parametrized by /3ii, /3i2, . • • , Pss- Thus, the number 
of parameters is d = S"^ and the number of transitions is m = 5'^. Model 
Q is represented by the x S'^ matrix whose rows are indexed by 

[S]'^ and the columns are indexed by words in [Sf. The entry of 

indexed by row aia2 G [5]^ and column w = [si, . . . , st) G [S]"^ is equal to 
\{i £ {I, . . . ,T - 1} \ crifT2 = SiSi+i } |. 

Example 2.2. Ordering [5]^ and [S]'^ lexicographically, the matrix A^ is: 
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2.3. Model Q. For Model (|c]), we consider the state space [•S'J^j;^. This 
model is parametrized by the positive real variables given by 71 , . . . , 75 and 
/3i25 /3i3, • • • , /3i,5, /321, hz-, • • • ) Ps-i,s- Thus, the number of parameters is 
d = S"^ and the number of transitions of state is m = Model ([c]) is 

the toric model represented by the S x S{S - 1)^-1 matrix A% defined 
below. The rows of A^j. are indexed by [S"] U [S*]^ \ { (i, i) | i G [S] } and the 
columns are indexed by words [5"]^^. 

(1) The entry of ^flp indexed by row s G [S] and column w = (si, . . . , st) G 
[S]^i^ is 1 if s = si and else. 

(2) The entry of indexed by row aia2 G [S]^, where ui 7^ (T2, and 

column w = (si, . . . , st) G [-^J^^. equal to | { i G {1, . . . , T — 1} | (T1CJ2 = SjSj+i } 

Example 2.3. For5 = 3 andT = 4, after ordering [S]yj[Sf\{ {i,i) \ i G [S]} 
and [Sj'j^i^ lexicographically, the matrix A^^ is: 
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2.4. Model Q. Lastly, Model (|d]) is parametrized by 5(5— 1) positive real 
variables. That is, it is parametrized by /3i2, /3i2, • • • , /^i.s, /32i, /323, • ■ • , Ps-i,s- 
Thus, the number of parameters is d = 5(5 — 1) and the number of transi- 
tions is m = I [5]^^| . Model (|d]) is the toric model represented by the 5(5 — 

l)x5(5-l)^-i matrix whose rows are indexed by [5]^\{ (i, i) | i G [5] } 
and the columns are indexed by words [5]^^;^. The entry of indexed 
by row criO"2 G [5]^, where 0"i / a2, and column w = {si, . . . , st) G [5']^^;^ is 
equal to | { i G {1, . . . , T - 1} | (Ji(T2 = SiSj+i } |. 

Example 2.4. Ordering [5]^ \ { \ i ^ [S] } and [5]^^ lexicographically 
and letting 5 = 3 and T = 4, the matrix is: 
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2.5. Sufficient statistics, ideals, and Markov basis. We refer to the 
matrices , AE,A^, and A^ as design matrices throughout this paper. 
Let A be A^ or ^0, for w ^ [S]^ we denote the column of A indexed by 
w by As,t{w) or simply by Aw when S and T are understood. Thus, by 
extending linearly, the map A : V([S']^) — t- M' is well-defined, where I is 
either 5^ or S{S — 1) respectively, depending on the model. 

Similarly let w G [S]%^, and A be A& or A^. We let As,t{w) or An, 
denote the column of A indexed by w. Again, we extend the map linearly 
so that A : ¥([5"]^^) is well-defined, where I is either 5^ or S{S - 1) 

depending on the model. 

Let W = {wi,..., wn} e 'P*{[Sf) (G 'P*i[S]JfL resp.) and we regard 
W as observed data which can be summarized in the data vector u G N'^ 

resp.) We index u by words in [S]'^ ( or [5"]^^ resp.), and 
Uw is equal to the number of words in W equal to w. Let A be one of the 
design matrices A^, A&, A^, or A&; then, as A is linear, then ^(u) 
is well-defined. We also adopt the notation, ^(W^) := ^(u). For W from 
V*{[S]'^) or 'P*{[S]Jfi^) (depending on the model) with data vector u, then 
is the sufficient statistic for the corresponding model. Often the data 
vector u is also referred to as a contingency table and are referred to 
as the marginals. (For a proof of sufficient statistics for Model Q see Hara 
and Takemura[5j and a proof of sufficient statistics for Model (Ibf see Hara 
and Takemurap].) 

The design matrices A^ and aE above define two toric ideals which are 
of our interest, as their set of generators are in bijection with the Markov 
bases of these models. For the Model (|b]), let /_4jb) be the toric ideal defined 
as the kernel of the homomorphism of polynomial rings tp : ]K[{p(w) | w G 
[S]^ }] K[{ Pij I i, i G [S] }] defined by ip : p{w) i-> I3sis2 ■ ■ ■ Pst-ist^ where 
K is a field and {p(w) | w G [5*]-^} is regarded as a set of indeterminates. 
Similarly, the toric ideal lyij^ corresponding to Model Q is defined as the 
kernel of the homomorphism ip' : ]K[{p(w) | w G [S']"'" }] — )• | i,j G 

['S']}][{7fc I k G [S]}] defined by Tp' : p{w) H> 'ys^PsiSi ■ ■ ■ f^sr-^sr, where 
again {p(w) | w G [S]'^ } is regarded as a set of indeterminates. 

The design matrices ^§ and A& from Model (|cj) and Model also 
define two toric ideals, which can be defined in a similar way as those from 



DEGREE BOUNDS FOR A MINIMAL MARKOV BASIS FOR THMC 



7 



models Q and Q respectively, or they can also be regarded as a special- 
ization of the ideals Ij^q and /_4jdj respectively, when we set Pa = for all 
iG[S]. 

Let A S ^'^x™- be a design matrix for one of om models. Then the set 
of all contingency tables (data vectors) satisfying a given marg inals b e 
is called a fiber which we denote by J-"b = { x E | ^(x) = b }. A move 
z E is an integer vector satisfying A{z) = 0. A Markov basis for a model 
defined by the design matrix A is defined as a finite set of moves Z satisfying 
that for all b and all pairs x, y S Th there exists a sequence zi, . . . ,zk & 2 
such that 

K I 
y = x + ^Zfc, x+y^Zfc > 0, l = l,...,K. 

k=l k=l 

A minimal Markov basis is a Markov basis which is minimal in terms of 
inclusion. See Diaconis and Sturmfels[3] for more details on Markov bases 
and their toric ideals. 

2.6. State Graph. Given any multiset W G 'P*{[S]'^) we consider the di- 
rected multigraph called the state graph G{W). The vertices of G(W) are 
given by the states [S] and the directed edges i ^ j are given by the tran- 
sitions from state i to j in w E W. Thus, we regard w £ W as a path of 
length T — 1 in G{W). See Figure[l]for an example of the state graph G{W) 
of the multiset of paths with length 3. W = {(112), (223), (331)} where 
[S] = {1,2,3} and T = 3. However, notice that the state graph in Figure [l] 
is the same for another multiset of paths W = {(122), (233), (311)}. 




Figure 1. The state graph G{W) oi W = 
{(112), (223), (331)}. Also the state graph G(W) of 
W = {(122), (233), (311)}. 



Proposition 2.5. 

(1) Let_W,W G V*{[Sf). A^{W) = A^{W) if and only if G{W) = 
G{W)._ 

■ Let W,W e V*{[S]%j^). A^{W) = A&iW) if and only ifG{W) = 
G{W). 
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The equivalence of the state graphs is not sufficient to show that W and 
W are in the same fiber when the initial states (Model ([a]) and Model (|c])) 
are considered. We extend the definition of the state graph to incorporate 
the initial states as follows: Let W G V*{[S]'^) and define the marked state 
graph G{W) to be the same as G{W) but with the additional condition that 
every vertex v of G{W) is marked with the number of words w £ W that 
start at state v. We illustrate this definition in Figure [2j 




Figure 2. The marked state graph G{W) of = 
{(11223), (33445), (55661)} G V{[S]'^). The vertices {1,3,5} 
are marked once since they are the initial states of the paths 
in W. 



Proposition 2.6. 

(1) LetJW,W G V*{[SY). A^{W) = A^{W) if and only ifG{W) = 
G{W)._ _ _ 

(2) Let W,W e V*{[S]Ijl). A^{W) = A^{W) if and only ifG{W) = 
G{W). 

If i is a vertex of the (possibly marked) multigraph G (or G), then we let 
Gj+ denote out-degree of i, which is the number of directed edges leaving 
vertex i. Similarly we let G+j denote the in- degree of i, which is the number 
of directed edges entering vertex i. 

3. Smith Normal Form 

For an integer matrix A G Z'^^'", we consider the Smith normal form 
D of A, which is a diagonal matrix D for which there exists unimodular 
matrices U G Z'^'"^ and V G Z™^™, such that UAV = D. The Smith 
normal form encodes the Z- module structure of the abelian group ZA := 
{niAi + • • • + UmAyn I nj G Z}. Some additional material about the Smith 
normal form for matrices with entries over a PID can be found in the book 
of C. Yap[llj. In this section, we explore the Smith normal form when the 
matrices to consider are those design matrices of the THMC model from the 



DEGREE BOUNDS FOR A MINIMAL MARKOV BASIS FOR THMC 



9 



previous sections. This study will be important for Sections [4] and [5| where 
we study normality of the toric ideal associated to the model. 

Our first result is the characterization of the Smith normal form for the 
design matrix of the model (|b]). 



Proposition 3.1. For any S and T, the Smith normal form of the design 
matrix is D = diag(l, . . . , 1, T — 1). 

Proof. For S and T fixed, let be the design matrix. We order the 
columns of so that the first 5^ columns correspond to paths of the 
form 11 . . . IsiSj for Sj,Sj G [S] ordered lexicographically. To simplify the 
notation in this proof, when w = 11 . . . IsjSj, we write only w = {siSj), 
as the prefix 11 ... 1 is understood. For example, when S = 3, the first 9 
columns of A^ are: 





(11) 


(12) 


(13) 


(21) 


(22) 


(23) 


(31) 


(32) 


(33 


11 


T-1 


T-2 


T-2 
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r-3 
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33 


























1 



We will use this example throughout this proof, in order to illustrate the 
arguments of the proof. 

Now, we apply the row operation that add all other rows to the first row. 
This row operation is encoded by a unimodular matrix U that has ones in 
the first row and in the diagonal, and zero in all other entries. 



/ 1 


1 


1 •• 


• 1 


1 \ 





1 


•• 


• 











1 •• 


• 
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. 1 





\^ 





•• 


• 
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For the case 5 = 3 from above, the first 9 columns of are: 

(11) (12) (13) (21) (22) (23) (31) (32) (33) 



11 
12 
13 
21 
22 
23 
31 
32 
33 



T-l 











T-l 
1 










T-l T-l T-l T-l T-l 




1 









1 



1 









1 






1 










1 





1 






T-l 


1 






1 





T-l 


1 







1 



Notice that the entries in the first row are all equal to T — 1, since the 
column sum of the design matrix is precisely T-l. Now, for every s ^ 1, 
we subtract the column (l,s) from the columns {s,Sj) for all Sj E [S]. For 
instance, in our example, we subtract the column (12) from the columns (21), 
(22), and (23), and similarly the column (13) from the columns (31), (32), 
and (33), to get: 



11 
12 
13 
21 
22 
23 
31 
32 
33 



:ii) 



^12) (13) (21) (22) (23) (31) (32) (33) 



T-l 











T-l T-l 



1 












1 














1 














1 














1 

















1 



Then, we subtract the first column (11) to those columns indicated by 
(1, s) for s 7^ 1. In our example, we subtract the first column to the columns 
(12) and (13) to have: 

(11) (12) (13) (21) (22) (23) (31) (32) (33) 



11 
12 
13 
21 
22 
23 
31 
32 
33 



T-l 














1 













1 














1 














1 















1 



Lastly, we subtract the first column (11) from all the other columns. 
Since these last operations involved only columns, we can encode them with 
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UA^V 



a unimodular matrix V' . In this way, we can write 

T-1 q 

Is2.i A 

where Is^-i is the (S^ — 1) x (5*^ — 1) identity matrix and A is some (S^ 
1) X {S'^~^ — 5^) integer matrix. 

We can use the first S*^ columns, to bring uaE V to the form D 
diag(T — 1, 1, . . . , 1), so there exists a unimodular matrix V such that 

T-1 

Icj2_l 



UA^V 



D. 



□ 



Although the proof we just presented is, by far, not the shortest argument 
we could find, we consider this constructive proof was worth to present, as 
it shows the special structure of the unimodular matrix U. We present now 
an important result related to the integer lattice generated by the columns 
of the design matrix. 

Let be the integer lattice generated by the columns of A&, i.e. 



w. 



Z^0 = i 

where A^ is the column of aB corresponding to the path 

Lemma 3.2. Let A^ be the design matrix for some S and T; then, y = 
(ui, . . . ,^52)^ G Z^IB if and only if yi + ■ ■ ■ + ys^ = mod (T — 1). 

Proof. Let D be the Smith normal form of A^ , and U, V un imodular matri- 
ces such that UA^ y = D, as in the proof of Propositions 



we write y = for some z G Z'^ 

unimodular 

{F"^z|z G Z^^} =Z' 
(zi, . . . ,%t)^. Then, 

/ (^-1)^1 \ 

Uy ~ 



3.1 



For y G 



Then y = U-^DV-^z. Since V'^ 



IS 



Write z := y ^z 



Dz 



V 



Z52 



Furthermore, we have the following integer system of equations 

2/1 + y2 H h ?/52 = (T - l)zi 

(3.1) 



2/2 



Z2 



2/52 = ZgT. 

The result follows from this system of equations. 



□ 
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We now prove an analogous result for Model For this, we will 

distinguish some pairs of paths {P, Q) whose transition count is the same 
except for one, as explained in the Remark |3.5| 



Definition 3.3. Let > 3, T > 4, and k £ [S] be three states which are 
pairwise distinct. Define the pairs of paths 



(3.2) 



(3.3) 



^kj ' Qkj 



{{{ijij ■ ■ ■ jik)},{{ikijij ■ ■ ■ ijij)} 
{{{ikjij ■ ■■jik)},{{ikikjij ■ ■ ■ jiji)} 



{{{kijij ■ ■ ■ ijij)}, {{k jiji ■ ■ ■ ijij)}) 
{{{kikjij ■ ■ • jiji)},{{kjikji • • • ijij)}) 



ifT even 
if T odd 



ifT even 
ifT odd 



232121 according to (3.2) 



and P^i 
For T = 
and P|f 



Example 3.4. Conforming with Definition 3.3, for S" > 3 and i = 2, j 
1, k = 3; we have: 

• When T = 6; then, P^^ = 212123, QH 

= 321212, Qlj = 312121 according to ( [3^ . 
7; then, P^^ = 2312123, 
= 3231212, Qll 

Remark 3.5. Let A be any of the design matrices for Models ([c]) and ([d]). 
Recall that if w is a path, then A{w) denotes the column of A indicated by 
w. We then observe the following: 



2323121 according to ( |3.2[ ) 
3123121 according to (|3.3|). 



(1) If Pll and Qj^- are as in (3.2), and we let v 



A{PP 



ki t 



-A{Ql^; then, 



-1 and Vst = for all other s,t £ 



^(pg)-^(gf' 



V satisfies vji = 1, Vki 
(2) If P^j and Q^} are as in ( |3.3[ ), and we let v — y^y± f^- , - ^y^i^jj 
then, V satisfies v^i = 1, v^j = —1 and Vgt = for all other s,t £ [S]. 
This is how our notation indicates, for instance, where are the two nonzero 
entries of the difference A{Pll) — A{Q'^^-); even more, it indicates that the 
{ji)th coordinate of this difference is 1 and the {ki)th coordinate is -1. 

Proposition 3.6. For S > 3 and T > A, the Smith normal form of the 
design matrix is D = diag(l, . . . , 1,T — 1). 

Proof. First we show that for all i,j G [S] such that / {1;2}, there 

exists a path w S [5*]^^ for which the column using only column 

operations, can be brought to the form v where v G 'L^^^~^\ with vi2 = 1, 
Vij = —1, and Vst = for all other s, t G [S]; we call the path w a pivot path. 
We will study four cases for the vector v, depending on the states i,j G [S] 
for which va = —1. 



1) Case i = 1, j > 2: Let the pivot path w be Pl^ from (3.3) of Def- 



inition 
s,te [S]. 



3.3 
TTT 



Then, as indicated in Remark 3.5 
satisfies vi2 = 1, vij = —1, and Vst 



A^iPlf) 
for all other 
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(2) Case i > 2, j = 1: Let k G [S] satisfying k ^ 1 and k ^ i. In 
this case, let the pivot path be if = P^l from (3.3) of Definition 
||3l Then, v = A&{P^i) + A^{P^^) + A^iP^^) - ^^(Qii) - 
A&iQl'y) - A&iQll) satisfies vu = 1, vn = -1, and Vst = oth- 
erwise. 



(3) Case i > 3, j 
Definition 3.3 

Vi2 = -1, and Vst 



= 2: Let the pivot path be tt; = 
Then, V = A^(Pli) - 



Pl^ from (lO) of 



i2 . 



satisfies vi2 = 1, 



for aU other s,t e [5]. 



(4) Case i > 2, j > 3: In this case we cover all other cases not covered 
before. For this, we let the pivot path he w = P^^ from ( |3.2| ) of 

A^{Pl^) + A^{Pl]) - A^iQlj) - 



Definition 

12^1 



3.3 



Then, v 



satisfies vi2 = 1, 



1-3 

-1, and V 



St 



otherwise. 



There are S{S — 1) — 1 pivot paths listed above; ordering these columns 
to be first in A^ and using the column operations from above, the design 
matrix A& can be brought into the form 



1 

-1 







V 



1 1 



-1 

-1 








1 




Ai Ai 




-1 



+1 



where Ai, ■ ■ ■ , A^ are unmodified columns of A^. The column Ai has non- 
negative integral entries that sum to T — 1. By adding integer multiples 
of the first ±1 vectors in the left of the modified A^ matrix, Ai can be 
transformed to (T — 1, 0, ... , 0)^. That is, A^ can be transformed by only 
column operations into 



1 

-1 







V 



1 1 



-1 

-1 








1 T-1 



Ai+i 



Ar. 



J 



By adding all the rows, besides the first, of the above matrix to the first 
row, and subtracting the column {T — 1,0,..., 0)^ from all the columns to 



14 



DAVID HAWS, ABRAHAM MARTfN DEL CAMPO, AND RURIKO YOSHIDA 



its right we get 

/ 
-1 



V 











-1 








T- 1 

_ 




\ 



^7- 



These operations can be encoded by unimodular matrices U and V: 



UA^V 



T-1 






S{S'-l)-l 



q 
A 



where U encode the row operation described above, so it is of the form: 



U 



1 1 


1 


1 •• 


• 1 


1 \ 





1 


•• 


• 











■ 


. 











•• 


• 1 










•• 


• 


1 ) 



The rest of the proof is the same as the one of Proposition 3.1 □ 

Lemma 3.7. y = (yi, . . . ,1/32)'^ £ if and only if yi + ■ ■ ■ + ys^ = 

mod (T-1). 



Proof. Since in our construction we get the same unimodular matrix U as 
in Proposition 3.1 then the same proof of Lemma 3.2 applies. □ 

4. Semigroup 

As studied in the last section, to an integer matrix A £ ^'^x'^ we associate 
an integer lattice ZA = {niAi + - ■ ■+nmAm \ ni G Z}. We can also associate 
the semigroup := {niAi + • • • + rimAm | G N}. We say that the 
semigroup NA is normal when x E NA if and only if there exist y E and 
a G M>g such that x = Ay and x = Aa. See Miller and SturmfelsfT] for 
more details on normality. 

In this section we will discuss the normality /non-normality of the semi- 
group generated by the columns of the design matrix for each model; A& , 
A^, A^, A^. 

4.1. Model ([a]). 

Lemma 4.1. The semigroup generated by the columns of the design matrix 
A^ is not normal for S > 3 and T > 4. 

Proof. First, for S = 3 and T = 4, we look at the semigroup generated 
by the columns of ^0. We ordered the indices of the coordinates lexico- 
graphically, i.e., as 1, 2, 3, (11), (12), (13), (21), (22), (23), (31), (32), (33). 
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We claim that the vector h := (1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 2, 1)^ is indeed not in 
the semigroup , as one can verify that there does not exist an integral 

non-negative solution for the system 

^0x = h, x> 0. 

. However, we have 

h - ^ i0 + ^ i0 + ^ i0 + ^ i0 

" ~ 2 (1112) ^ 2 (1232) ^ 2 (3232) ^ 2 (3332) ' 

Thus h is in the saturation of N^lil . Also since 
h=i0 +i0 

•^(1123) ^ "^(3332) ^ "^(3232) "^(3323)' 

we know that h is in the lattice generated by the columns of the matrix 
Thus h is in the difference between and its saturation. 

Based on this case, we show now that for T > 4, there is a vector in the sat- 
uration of N^0 but not in NA^. Let h := (1, 0, 1, T-3, 1, 0, 0,0,1, 0, 2, T- 
3)^ and notice that 

h-ii0 +-A^ +-A^ +-A^ 

2 (1---1112) 2 (1---1232) 2 (3^--3232) ^ 2 (3^--3332)' 

where (1 . . . 1112) is the path with T — 4 many I's in front of the path 
(1112), (1 . . . 1232) is the path with T — 4 many I's in front of the path 
(1232), (3 . . . 3232) is the path with T — 4 many 3's in front of the path 
(3232), and (3 . . . 3332) is the path with T — 4 many 3's in front of the path 
(3332). Thus h is in the saturation of N^0. Also 

h = (1, 0,1, r-3,1, 0,0,0,1, 0,2,r-3)^ 
= -A^ 

•^(1...1123) ^ "^{3.. .3332) ^ "^{3.. .3232) "^{3. ..3323) 

= (i,o,o,r-3,i, 0,0,0,1, 0,0,0)^ 
+ (o,o,i,o,o,o,o,o,o,o,i,r-2)T 

+ (0,0, 1,0, 0,0, 0,0, 1,0,2, r-4)"^ 

- (0,0, 1,0, 0,0, 0,0, 1,0,1, r- 3)"^. 

Thus h is in the integer lattice ZA^ but there does not exist an integral 
non-negative solution for the system 

^0x = h, x>0. 

Thus h is not in N^il. 

For 5 > 3, we just observe that "ZA^ Q ZA^-^ rp. □ 

When 5 = 2, the semigroup seem to be normal, as stated in the 

following conjecture. 

Conjecture 4.2. The semigroup generated by the columns of the design 
matrix A^ is normal for 5 = 2 and T > 2. 

We verified this conjecture using the software nornializ[2] for T = 1, . . . , 100. 
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4.2. Model (|b]). Different from Model (|a|, the Model (|bj) does not satisfy 
the normality condition. 

Lemma 4.3. The semigroup generated by the columns of the design matrix 
is not normal for S > 2 and T > 3. 

Proof For r > 3 and S = 2, let h = (1, 0, 0, T - 2)"^. We want to show that 
h is not in the semigroup but it is in the saturation of . 

Note that h can be written as 

^ (r-i,o,o,o)T+^(o,o,o,r-i)T = _l_^0^ n+^-^ft 



_ '""'"'"z _ -[^ "1 "1 2^ _ 2^'^(lll...l)^2^ _ ]^'^{222...2)' 

where (111 ... 1) is the path of T many I's, and (222 ... 2) is the path of T 
many 2's. We can also write h as 

(T - 1,0, 0, 0)^ - (T - 2, 1, 0, 0)^ + (0, 1, 0, T - 2)^ 

~ •^(T;i,...,i,i) •^(T;i,...,i,2) ^ •^(T;2,...,2,2) 

where (1,1,..., 1, 1) is the path of T — 1 many I's, (1,1,..., 1, 2) is the path 
of T — 2 many I's and one 2, and (1,2,..., 2, 2) is the path of T — 2 many 2's 
and one 1. Thus h is in the lattice generate by the columns of . Thus 
h is in the saturation of and in the integer lattice ZA^, but there 

does not exist an integral non-negative solution for the system 

A^x = h, X > 

because h consist of only one transition of the form 11, and T — 2 transitions 
of the form 22. Thus h is not in but h is a lattice point in the cone 

generated by the columns of A^ . 

For 5 > 2, we just set all the transitions involving the state s > 2, s G [5] 
to be zero. □ 

4.3. Model (|c]). For S = 3 and for T = 4, ... ,9, we have computed the 
Hilbert basis for the cone generated by the columns of the design matrix 
A^ over the lattice generated by the columns of the matrix using 
normaliz. The running time of normaliz was under two seconds for all 
data sets. The most time consuming part in our experiment was generating 
the design matrices. It turns out that the set of columns of A^ contains the 
Hilbert basis for all cases, which implies normality. See Table [T] in Section [6] 
for more details. Thus we have the following conjecture. 

Conjecture 4.4. For S = 3 and for T > 4, the semigroup generated by the 
columns of the design matrix A^ is normal. 

4.4. Model ([d]). Similarly, using normaliz, we have computed the Hilbert 
basis for cone generated by A^ over the lattice ZA& for T = 4, . . . , 15. 
The running time of normaliz was again under two seconds for all data sets. 
It turns out that the set of columns of A& contains already the Hilbert basis 
for all cases, which implies normality. In Table [2] of Section |6] we present 
these results, which support the following conjecture. 
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Conjecture 4.5. For S = 3 and for T > 4, the semigroup generated by the 
columns of the design matrix is normal. 



5. PoLYTOPE Structure 

We recall some necessary definitions from polyhedral geometry and we 
refer the reader to the book of Schrijver [9] for more details. The convex 
hull of {ai, . . . , a^} C M" is defined as 

{mm '^ 
X G M" I X = ^ Xia.i, ^ Aj = 1, Ai > > . 
i=l i=l ) 

A polytope P is the convex hull of finitely many points. We say P C P is a 
face of the polytope P if there exists a vector c such that F = arg maxxg p c • 
X. Every face P of P is also a polytope. If the dimension of P is d, a face P 
is a facet if it is of dimension d — \. For /c S N, we define the fc-th dilation 
of P as kP := { A;x I X E P, }. A point x G P is a vertex if and only if it can 
not be written as a convex combination of points from P\{x}. 

The cone of {ai, . . . , a^} C is defined as 

cone(ai, . . . , a^) := | x € M" | x = ^ Ajaj, Ai > | . 

We are interested in the polytopes given by the convex hull of the columns 
of the design matrices of our four models. If A is the design matrix for one 
of our four models, we will simply use A to refer to the set of columns 
of the design matrix when the context is clear. Let P0 = conv 
P0 = conv (^0), P§ = conv (^§), and P@ = conv (^0). Also, we 
let C0 = cone(^§), C0 = cone(^0), C§ = cone(^§), and C0 = 
cone (^0). 

In this section we will focus mainly on Model Q . If x G M'^^'^"^) , we index 
X by { I ^ <i,j < S, i ^ j}. We define Cij E M'^('^~^) to be the vector 
of all zeros, except 1 at index ij. We also adopt the notation Xi+ := ^ij 

and x+i := 'Ylj^ji- Fc"^ ^'^Y ^ ^ -^^g define a multigraph G(x) 

on S vertices, where there directed edges from vertex i to vertex j. 

One would like to identify the vectors x G ^'^'(•S'-i) fQj- which the graph G(x) 
is a state graph. Nevertheless, observe that Xi+ is the out-degree of vertex 
i and x+i is the in-degree of vertex i. 

Proposition 5.1. // x G Z^0 n C0 then Y^i^j^ij = - 1) for some 
k £N and \xi^ — x+i| < k for all i £ S. 



Proof. By Lemma 3.7, we have Yli^j ^ij ~ ~ 1) some k £ N. Let 
[Ai ■ ■ ■ Am] be the columns of the design matrix ^0 . Then x G C0 implies 
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X = YllLi o^i^i where ai > 0. Then 



i=l 



= ociUiWi = (T - 1) = k{T - 1). 



Thus YT=i tti = G N. Finahy for f G 5 

\xi+ - x+i\ = |ai(^i)i+ H h am{Am)i+ - ai(^i)+i am{Am)+i\ 

m 

= \ai [{Ai)i+ - {Ai)+i] +--- + am [(^m)i+ - {Am)+i\ \<Yai = k 

1=1 

since | [{Ai)i+ - {Ai)+i] | < 1 for all 1 < / < m. □ 

Proposition 5.1 states that for x G n the multigraph G{x) will 

have in-degree and out-degree bounded by ||x||i/(T — 1) at every vertex. 
This implies nice properties when S = 3 and ||x||i = (r — 1). Recall a path 
in a multigraph is Eulerian if it visits every edge only once. 

Proposition 5.2. If G is a multigraph on three vertices, with no self-loops, 
T — 1 edges, and satisfying 

\Gi+-G+i\<l i = 1,2,3; 

then, there exists an Eulerian path in G. 

Proof. First consider the case where Gj+ = G+j for i = 1,2,3. Then, since 
G contains no self-loops, the T — 1 edges of G consists of disjoint cycles of 
the form i^j^i or i^j^k^i, where i ^ j , i ^ k and j ^ k. Since 
G only has three vertices, every cycle has a vertex in common. Thus, there 
is an Eulerian path that visits each cycle. 

Suppose, without loss of generality, that Gi+ = G+i -|- 1 and G2+ + 1 = 
G+2. Then, there must exist edge 1 — ?■ 2 or the path 1 — t- 3 — t- 2. This follows 
since vertex 1 has more outgoing edges then incoming, and they must go to 
either 2 or 3. Similarly vertex 2 has more incoming edges than outgoing, 
and they must come from either 1 or 3. Let p he a path from vertex 1 to 
vertex 2 (either 1— )-2orl— J-S— >-2). 

Let G = G\p, that is, the graph G with the edge(s) of p removed. Note 
that G+i = G+2 = G2+, G+3 = G+3, and G3+ = G3-1-. Observe that 
G3+ = G4.3, as otherwise we would have a contradiction since G has one 
vertex with non-zero in-degree minus out-degree. Now, as in the first case, 
G consists of disjoint cycles of the form i^j^iori^j^k^i, where 
^ 7^ J) ^ 7^ ^ ^i^d J 7^ ^- Thus, there is an Eulerian path on G that visits 
each cycle of G, and we can append or prepend p to get an Eulerian path 
on G. □ 

Remark 5.3. Note that every wordw e V*{[S]%j^) gives an Eulerian path 
in G{{w}) containing all edges. Conversely, for every multigraph G with 
an Eulerian path containing all edges, there exist w G 'P* (,[S]Jf i) such that 
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Figure 3. For word 123131 G 'P*{[3]%j^), the state graph 
G({123131}) is given above. Note that there are multiple 
Eulerian paths (words) in the multigraph above. 



G{{w} )= G. More specifically, w is the Eulerian path in G{{w}). See Figure 

m 

Lemma 5.4. If S = 3 and T > 4, then n Z'^^'^'-i) = . 

Proof. Certainly P0 n D . Let x E P0 n Z^. Then ||x||i = 
mod T — 1 and Wi £ S we have — < 1. Finally, considering the 
multigraph G(x) and Proposition 5.2 we see that x is equal to some column 
ofA^. □ 



As demonstrated in Lemma 5.4 we will find it useful to consider x G 
|!^S'{S'-i) ^ vector, and also as a multigraph G(x). 
We define 



Hk 



X e 



k{T 



Proposition 5.5. 



(1) ForS>3,T>Aandke N, 

kP® = c@ n Hk. 

(2) For S>3 and T > 4, 

oo 

C0 n ZA^ = {kP& n z^(^-i)) . 

fc=0 

As we will be focusing on Model (|d]) with 5 = 3, we give a few definitions 
specific to this case. Let G be a directed multigraph on three vertices with 
T — 1 edges and no self- loops. We call a cycle i ^ j ^ i, where i ^ j, a 
two-cycle. Similarly we call a cycle i ^ j ^ k ^ i, where i ^ j, i ^ k, 
and j ^ k, a three-cycle. We say the two-cycles i ^ j ^ i and k ^ I ^ k 
in G have different type if {ij} / {kl}. We let G^ be the subgraph of G 
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G 




G^ 




G\G^ 



(G\G^)\G^ 



Figure 4. A graph G (from word 123131312) and its two- 
cycles (G^), its three-cycles (G^), G remove its two-cycles 
and G remove its two-cycles then three-cycles 
{{G\G')\G'). 



consisting of only the two-cycles of G. Similarly, we let G^ be the subgraph of 
G consisting of only the three-cycles of G. By G\G^, we mean the subgraph 
of G with the edges in G^ removed. Similarly for G\G^. We let |G| be the 
number of edges in G. We illustrate this in Figure |4j 
Let T > 1 and we define 

G ■= { G{{w}) I w € [3]^^, G{{w}) has only one type of two-cycle } , 



i>m,n 



Geo 



\G^ 



2\3| 



m, 



\{G\G') 



n 



2 ' 3 

Notice that the graph G in Figure [4] is in since it has only one type of 
two-cycle. Moreover, G is contained in ^2,1- 

Remark 5.6. If a multigraph G on three vertices and T — 1 edges with no 
self-loops has only one type of two-cycle, then every three-cycle must have 
the same orientation. See Figure^ 

Remark 5.7. Note that 

\Qm,n\ < 18 

for any m,n, and T . There are m two-cycles of the same type, hence three 
ways to place them. The n three-cycles must he the same orientation by 



Remark 5.6, hence two ways to place them. Finally, the remaining one or 



two edges must be placed in the same orientation as the three-cycles, hence 
three ways to place them. 
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Figure 5. The red and blue three-cycles in the multigraph 
on the left have opposite orientation, implying there are mul- 
tiple types of two cycles, red, green and yellow in the multi- 
graph on the right. Hence, this is not a vertex of 




1 

2 




Figure 6. On the left is the multigraph for [0 1 1 1 l]"*" G 
which can be written as a linear combination of vectors 



[0 2 2 0]' G Afi and [0 2 2]^ g ^| 



Lemma 5.8. Let 5 = 3 and T > 1 and x G be a vertex. If i ^ j ^ i 
and k ^ I ^ k are two-cycles in G(x), then they consist of the same edges. 
That is, {iV\ = {/c/}. 

Proof. We will prove the contrapositive. Suppose x G P0 n where 
i ^ j ^ i and k ^ I ^ k are two-cycles in G(x) such that they do not 
consist of the same edges. Then Xij > 1, Xji > 1, Xki > 1 and xm > 1- 
Moreoever {ij} Ci {kl} = 0. Let y = x + e ^; + e^^ — e^j — eji and z = 
X + eij + eji — Bki — eik. Note that, by Remark 5.3 , we must have y, z G P0 
since we are only removing and adding two- cycles. Then x=iy + iz. □ 



Lemma 5.8 can be proved by also considering convex combinations of 
multigraphs. See Figure [6| 

By definition of the convex hu ll, th e vertices of P& will be contained in 



the columns of A^ . By Lemma 5.8 for 5 = 3, the vertices of will be 



contained in Q, the set of directed multigraphs on three vertices that have 



only one type of two-cycle. It is not difficult to see that Q = |J,„ mez> 
Note that Gm n is non-empty depending on T, m and n. 
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Figure 7. The graph G on the left (from word 
1212121231231) is contained in G3,2, and by Theo- 
can be written as |G({23123123123123}) + 



rem 



5.10| 

iG({212121212123}). Hence, G is not a vertex of P0. 



For t e M, let 



(T-l-2t) 
3 





if < 2t < T - 1, 
otherwise. 



Proposition 5.9. Let S = 3 and T > 1. Then Qm,f{m) 7^ ^ for < 2m < 

T -1, else Qm,n = 0- 



Therefore, if p = [(T — l)/2j , we can write Q = Qq j/q) U • • • U & 



The 



3, all graphs (vectors) in 



main idea behind Theorem 5.10 is that for S 
Qm,n are not vertices for many (m,n). 

Theorem 5.10. Let S = 3. The number of vertices of is bounded by 
some constant G which does not depend on T. 

Proof. Let T > 13 and p = [{T - l)/2j . We have G = Goj(o) U • • • U Gpj(p). 
We now claim for 3 < g < p — 3, that every graph(vector) in Qqj[q) is not a 
vertex. Let x G Qqj(q) for 3 < g < p + 3. Note that \ f{q — 3) + \f{q + 3) = 
f{q). Let y be derived from x where two of the three-cycles are removed 
and three two-cycles are added such that y G ^q_3j(q_3). Similarly let z 
be derived from x where three two-cycles are removed and two of the three- 
cycles are added such that z G ^g_|_3j(q_|_3). Finally, x = + gz. We 
illustrate this construction in Figure [7| 

Thus, the vertices of must be contained in ^o,/(o) U ^ ^2,/(2) U 

Qp-2j(p-2)^Gp-ij{p-i)^Gpj(p)i which is bounded for all T. Note that each 
set of graphs in the above union is finite by definition. □ 



6. Computational Results 

Here we give two tables listing the f- vector and number of Hilbert basis 
elements for 5 = 3 and Model (|c]) and Q. The Hilbert basis and support- 
ing hyperplanes were computed with normaliz [2], and the f- vectors were 
computed using Polymake [4j. For Model ([c]) we computed the f- vector and 
Hilbert basis for S = 3 and T = 1, . . . , 9, shown in Figure [!} For Model @ 
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T 


#HB 


/o 


fl 


/2 


/3 


h 


h 


h 


4 


24 


24 


156 


434 


606 


444 


162 


24 


5 


39 


36 


249 


671 


891 


615 


210 


30 


6 


60 


42 


276 


689 


837 


528 


168 


24 


7 


87 


54 


351 


860 


1020 


633 


204 


30 


8 


120 


60 


372 


851 


939 


546 


168 


24 


9 


162 


72 


435 


968 


1062 


633 


204 


30 



Table 1. The number of Hilbert basis elements (normaliz) 
and f-vectors (Polymake) for Model ([c]) where S = 3. The 
running time of normaliz was under two seconds for all data 
sets. 
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20 
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12 
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27 
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41 


144 


189 


108 
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42 


171 


230 


123 


24 
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123 


45 


186 


245 


126 


24 


10 


166 


56 


201 


252 


129 


24 


11 


207 


63 


216 


257 


126 


24 


12 


264 


54 


189 


236 


123 


24 


13 


320 


77 


246 


279 


132 


24 


14 


396 


54 


189 


236 


123 


24 


15 


468 


63 


216 


257 


126 


24 



Table 2. The number of Hilbert basis elements (normaliz) 
and f-vectors (Polymake) for Model Q where S = 3. The 
running time of normaliz was under two seconds for all data 
sets. 



we computed the f- vector and Hilbert basis for S = 3 and T = 1, . . . , 15, 
shown in Figure ^ The supporting hyperplanes of C§ and C0 for S = 3 
computed by normaliz[2j are given in the Appendix. 



All supplementary material can be found at http://www.davidhaws, 



[net/Pro j ects/ToricMarkovChain/ Software to draw state graphs and 



move graphs can be found at https : //github . com/dchaws/DrawStateMoveGraphsj 

Software to generate all words, all words with no self- loops and the design 



matrices can be found at https://github.com/dchaws/GenWordsTrans 
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r = 3 


4 


5 


6 


r = 4 


4 


5 





Table 3. Cases where Conjecture |7.3| was tested 



7. Conclusions and Open Problems 

One notices that the set of columns is a graded set since there exists 
w G such that aj • w = 1 by Lemma 4.14 in Sturmfels|10j. 
One tool is coming from Theorem 13.14 in Sturmfels|10|. 



Theorem 7.1 (Theorem 13.14 in Sturmfels|10j). Let A <ZU^ he a graded set 
such that the semigroup generated by the elements in A is normal. Then the 
toric ideal I a associate with the set A is generated by homogeneous binomials 
of degree at most d. 



Using Theorem 5.10, Conjecture 4.5 and Theorem 7.1 one can prove the 
following theorem: 

Conjecture 7.2. We consider Model Q. Then for 5 = 3 and for any 

T > A, a Markov basis for the toric ideal /_4jd) consists of binomials of 
degree less than or equal to d = 6. Moreover, there are only finitely many 
moves up to a certain shift equivalence relation. 

On the experimentations we ran, we found evidence that more should be 
true. 



Conjecture 7.3. Fix S > 3; then, for every T > 4, there is a Markov basis 
for the toric ideal /_4jd) consisting of binomials of degree at most S — 1, and 
there is a Grobner basis with respect to some term ordering consisting of 
binomials of degree at most S. 

Despite the computational limitations (the number of generators grows 



exponentially when T grows,) we were able to test Conjecture 7.3 using 
4ti2[lJ for the following cases: 

Supporting Hyperplanes In this appendix, we present the supporting hy- 
perplanes of for S = 3 computed by normaliz[2]. Hyperplanes are given 
by column vectors [ci2 C13 C21 C23 C31 C32]''' where C12X12 + 0132:13 + 021^21 + 
C232;23 + C31X31 + 032X32 > 0. For all cases computed, the non-negativity 
constraints were given as hyperplanes and are not included for brevity. 
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In what follows, we present the supporting hyperplanes of for S = 
3 computed by normal iz[2l. Hyperplanes are given by column vectors 
[ci C2 C3 ci2 ci3 C21 C23 C31 €32]^ where cixi + C2X2 + 03X3 + C12X12 + 
Ci3a;i3 + C21X21 + 023^23 + C31X31 + C32X32 > 0. For ah cases computed, the 
non-negativity constraints were given as hyperplanes and are not included 
for brevity. The number of hyperplanes alternates between 22 and 30. 
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